Apache Hadoop Changelog

Release 0.20.1 - 2009-09-01

INCOMPATIBLE CHANGES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-5726 Remove pre-emption from the capacity scheduler code base Major . Hemanth Yamijala rahul k singh
HADOOP-5881 Simplify configuration related to task-memory-monitoring and memory-based scheduling Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli

NEW FEATURES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-5714 Metric to show number of fs.exists (or number of getFileInfo) calls Minor metrics Koji Noguchi Jakob Homan
HADOOP-3315 New binary file format Major io Owen O’Malley Hong Tang
HADOOP-6080 Handling of Trash with quota Major fs Koji Noguchi Jakob Homan

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
MAPREDUCE-465 Deprecate org.apache.hadoop.mapred.lib.MultithreadedMapRunner Minor . Amareshwari Sriramadasu Amareshwari Sriramadasu
HDFS-527 Refactor DFSClient constructors Major hdfs-client Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
MAPREDUCE-767 to remove mapreduce dependency on commons-cli2 Major contrib/streaming Giridharan Kesavan Amar Kamat
HDFS-635 HDFS Project page does not show 0.20.1 documentation/release information. Major documentation Andy Sautins

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-5210 Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high Minor . Jothi Padmanabhan Ravi Gummadi
HADOOP-5654 TestReplicationPolicy.<init> fails on java.net.BindException Major test Hairong Kuang Hairong Kuang
HADOOP-5655 TestMRServerPorts fails on java.net.BindException Major . Hairong Kuang Devaraj Das
HADOOP-5533 Recovery duration shown on the jobtracker webpage is inaccurate Major . Amar Kamat Amar Kamat
HADOOP-5646 TestQueueCapacities is failing Hudson tests for the last few builds Major . Jothi Padmanabhan Vinod Kumar Vavilapalli
HADOOP-5691 org.apache.hadoop.mapreduce.Reducer should not be abstract. Major . Amareshwari Sriramadasu Amareshwari Sriramadasu
HADOOP-5688 HftpFileSystem.getChecksum(..) does not work for the paths with scheme and authority Major . Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-4674 hadoop fs -help should list detailed help info for the following commands: test, text, tail, stat & touchz Trivial fs David NeSmith Ravi Phulari
HADOOP-5711 Change Namenode file close log to info Minor . Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-5213 BZip2CompressionOutputStream NullPointerException Blocker io Zheng Shao Zheng Shao
HADOOP-5736 Update CapacityScheduler documentation to reflect latest changes Major . Sreekanth Ramakrishnan Sreekanth Ramakrishnan
HADOOP-5718 Capacity Scheduler should not check for presence of default queue while starting up. Major . Sreekanth Ramakrishnan Sreekanth Ramakrishnan
HADOOP-5719 Jobs failed during job initalization are never removed from Capacity Schedulers waiting list Major . Sreekanth Ramakrishnan Sreekanth Ramakrishnan
HADOOP-5349 When the size required for a path is -1, LocalDirAllocator.getLocalPathForWrite fails with a DiskCheckerException when the disk it selects is bad. Major . Vinod Kumar Vavilapalli Vinod Kumar Vavilapalli
HADOOP-5636 Job is left in Running state after a killJob Critical . Amareshwari Sriramadasu Amar Kamat
HADOOP-5641 Possible NPE in CapacityScheduler’s MemoryMatcher Major . Vinod Kumar Vavilapalli Hemanth Yamijala
HADOOP-5828 Use absolute path for JobTracker’s mapred.local.dir in MiniMRCluster Major test Hemanth Yamijala Hemanth Yamijala
HADOOP-5850 map/reduce doesn’t run jobs with 0 maps Critical . Owen O’Malley Vinod Kumar Vavilapalli
HADOOP-4626 API link in forrest doc should point to the same version of hadoop. Minor documentation Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-5883 TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory Major . Hemanth Yamijala Hemanth Yamijala
HADOOP-5539 o.a.h.mapred.Merger not maintaining map out compression on intermediate files Blocker . Billy Pearson Jothi Padmanabhan
HADOOP-5932 MemoryMatcher logs 0 as freeMemOnTT even though there are free slots available on TaskTraker Major . Karam Singh Vinod Kumar Vavilapalli
HADOOP-5951 StorageInfo needs Apache license header. Major . Suresh Srinivas Suresh Srinivas
HADOOP-5648 Not able to generate gridmix.jar on already compiled version of hadoop Major benchmarks Suman Sehgal Giridharan Kesavan
HADOOP-5908 ArithmeticException in heartbeats with zero map jobs Major . Vinod Kumar Vavilapalli Amar Kamat
HADOOP-5924 JT fails to recover the jobs after restart after HADOOP:4372 Major . Ramya Sunil Amar Kamat
HADOOP-5882 Progress is not updated when the New Reducer is running reduce phase Blocker . Jothi Padmanabhan Amareshwari Sriramadasu
HADOOP-5746 Errors encountered in MROutputThread after the last map/reduce call can go undetected Major . Devaraj Das Amar Kamat
HADOOP-5884 Capacity scheduler should account high memory jobs as using more capacity of the queue Major . Hemanth Yamijala Vinod Kumar Vavilapalli
HADOOP-5937 Correct info message “Use hadoop dfs -safemode option” to " Use hdfs dfsadmin -safemode option" . Minor . Ravi Phulari Ravi Phulari
HADOOP-5921 JobTracker does not come up because of NotReplicatedYetException Major . Amareshwari Sriramadasu Amar Kamat
HADOOP-6017 NameNode and SecondaryNameNode fail to restart because of abnormal filenames. Blocker . Raghu Angadi Tsz Wo Nicholas Sze
HADOOP-5920 TestJobHistory fails some times. Major . Amareshwari Sriramadasu Amar Kamat
HDFS-26 HADOOP-5862 for version .20 (Namespace quota exceeded message unclear) Major . Boris Shkolnik Boris Shkolnik
HDFS-438 Improve help message for quotas Minor . Raghu Angadi Raghu Angadi
MAPREDUCE-2 ArrayOutOfIndex error in KeyFieldBasedPartitioner on empty key Major . Amar Kamat Amar Kamat
MAPREDUCE-130 Delete the jobconf copy from the log directory of the JobTracker when the job is retired Major . Devaraj Das Amar Kamat
MAPREDUCE-657 CompletedJobStatusStore hardcodes filesystem to hdfs Major jobtracker Amar Kamat Amar Kamat
MAPREDUCE-179 setProgress not called for new RecordReaders Blocker . Chris Douglas Chris Douglas
MAPREDUCE-124 When abortTask of OutputCommitter fails with an Exception for a map-only job, the task is marked as success Major . Jothi Padmanabhan Amareshwari Sriramadasu
HADOOP-6139 Incomplete help message is displayed for rm and rmr options. Minor . Suman Sehgal Jakob Homan
HADOOP-6141 hadoop 0.20 branch “test-patch” is broken Major build Hong Tang Hong Tang
HADOOP-6145 No error message for deleting non-existant file or directory. Major fs Suman Sehgal Jakob Homan
MAPREDUCE-565 Partitioner does not work with new API Blocker task Jothi Padmanabhan Owen O’Malley
MAPREDUCE-18 Under load the shuffle sometimes gets incorrect data Blocker . Owen O’Malley Ravi Gummadi
MAPREDUCE-735 ArrayIndexOutOfBoundsException is thrown by KeyFieldBasedPartitioner Major . Suman Sehgal Amar Kamat
MAPREDUCE-383 pipes combiner does not reset properly after a spill Major . Christian Kunz Christian Kunz
MAPREDUCE-40 Memory management variables need a backwards compatibility option after HADOOP-5881 Blocker . Hemanth Yamijala rahul k singh
MAPREDUCE-796 Encountered “ClassCastException” on tasktracker while running wordcount with MultithreadedMapRunner Major examples Suman Sehgal Amar Kamat
MAPREDUCE-838 Task succeeds even when committer.commitTask fails with IOException Blocker task Koji Noguchi Amareshwari Sriramadasu
MAPREDUCE-805 Deadlock in Jobtracker Major . Michael Tamm Amar Kamat
HDFS-167 DFSClient continues to retry indefinitely Minor hdfs-client Derek Wollenstein Bill Zeller
MAPREDUCE-832 Too many WARN messages about deprecated memorty config variables in JobTacker log Major . Karam Singh rahul k singh
MAPREDUCE-745 TestRecoveryManager fails sometimes Major jobtracker Amareshwari Sriramadasu Amar Kamat
MAPREDUCE-834 When TaskTracker config use old memory management values its memory monitoring is diabled. Major . Karam Singh Sreekanth Ramakrishnan
MAPREDUCE-818 org.apache.hadoop.mapreduce.Counters.getGroup returns null if the group name doesnt exist. Minor . Amareshwari Sriramadasu Amareshwari Sriramadasu
MAPREDUCE-807 Stray user files in mapred.system.dir with permissions other than 777 can prevent the jobtracker from starting up. Blocker jobtracker Amar Kamat Amar Kamat
MAPREDUCE-430 Task stuck in cleanup with OutOfMemoryErrors Major . Amareshwari Sriramadasu Amar Kamat
HADOOP-6215 fix GenericOptionParser to deal with -D with ‘=’ in the value Major . Owen O’Malley Amar Kamat
MAPREDUCE-421 mapred pipes might return exit code 0 even when failing Major pipes Christian Kunz Christian Kunz
HDFS-525 ListPathsServlet.java uses static SimpleDateFormat that has threading issues Major namenode Suresh Srinivas Suresh Srinivas
MAPREDUCE-911 TestTaskFail fail sometimes Major test Amareshwari Sriramadasu Amareshwari Sriramadasu
MAPREDUCE-687 TestMiniMRMapRedDebugScript fails sometimes Major test Amar Kamat Amareshwari Sriramadasu
MAPREDUCE-924 TestPipes must not directly invoke ‘main’ of pipes as an exit from main could cause the testcase to crash. Major pipes Amareshwari Sriramadasu Amareshwari Sriramadasu
HDFS-1022 Merge under-10-min tests specs into one file Major test Erik Steffl Erik Steffl

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-6213 Remove commons dependency on commons-cli2 Blocker util Amar Kamat Amar Kamat